PR description:

This PR proposes an evolution of the "gpu framework" for Alpaka EDModules and ESModules. The current model (introduced in #38855) is similar to the model we have for CUDA EDModules, whereas this development builds on earlier prototypes (in cms-patatrack/pixeltrack-standalone#224, cms-patatrack/pixeltrack-standalone#256, cms-patatrack/pixeltrack-standalone#257 for CUDA, and in cms-patatrack/pixeltrack-standalone#314 for Alpaka). The new model was presented in the HLT GPU developments meeting in https://indico.cern.ch/event/1192252/#4-new-cmssw-accelerator-framew .

The two models can coexist. The EDModules/ESModules of the current model can not access the device data products of the new model. I believe (did not test) the new model EDModules would be able to use the ESProducts of the current model, and if really necessary, they could be made to use the EDProducts of the current model (but I'd prefer to avoid that).

Shortcomings of the CUDA model that this PR addresses are

• Cumbersome interface (many reasons are listed in Improve CUDA EDProducers #30266)
  • New interfaces resemble the interfaces elsewhere in the framework
• ScopedContext* destructor does not check for errors in CUDA API calls in order to avoid exception being thrown from a destructor
  • Now these functions are called from normal functions (in a base class)
• Situation where a CUDA EDProducer produces at least two device products that are consumed by at least two different consumers can lead to situation where one CUDA stream is shared by independent branches of the DAG (described in more detail in [cudadev][RFC] Next version of CUDA EDModule API cms-patatrack/pixeltrack-standalone#224)
  • The "metadata" object holding the Queue is now shared across all device products of an EDProducer
• Requires CUDA runtime API calls in the EventSetup data formats, spreading the CUDA dependence wide in CMSSW
  • All device API functions are now called in ESProducer, data formats only hold the data. The device runtime dependence is still there though (because the memory buffers have explicit dependence), but limited to the shared libraries of the corresponding backends
• Effectively prevents kernel calls in ESProducers to produce some of the data, and an ESProducer to consume device data
  • Both are now allowed
• In the EventSetup system, the transferred-from-host memory is kept alive for the entire duration of the IOV, whereas the memory could be released as soon as all transfers have finished (if the host-memory product is not consumed for anything else)
  • The model here allows an earlier (pinned) host memory release via the use of edm::ESTransientHandle. We'd still need to enable the actual memory optimization (Look into enabling edm::ESTransientHandle memory optimization again #31085)
• EDModules check on every event if the ESProduct data is ready
  • Now the ESProducts are enforced to be fully ready before consuming modules are run

The device product wrapper, that is currently cms::cuda::Product<T> / cms::alpakatools::Product<TQueue, T> is now made part of the framework as edm::DeviceProduct<T>. Making it a framework type allows some future developments that would be difficult without. Currently the backend/memory space specific metadata are included in the edm::DeviceProduct<T> in a type-erased way, such that only the correct backend code can access the data. This may stay or change in the future.

This PR makes the host-synchronous backend(s) to produce the T directly instead of wrapping it to edm::DeviceProduct<T> (which was left as future improvement in #38855).

The PR explores three possible models for ESProducts (which are indicated in comments in the code, these were also summarized in the slides linked above)

1. The product class is fully defined inside ALPAKA_ACCELERATOR_NAMESPACE
2. The product class is defined as a template with Device template argument
3. The product class is defined as PortableCollection

The PR currently has three two limitations compared to the current models

1. The automatic transfers from device to host are synchronous. The automatic transfer from device to host are now asynchronous (with the aid of Added transformAsync ability to modules #39557)
   • This shortcoming will be improved in a future PR after an asynchronous version of Transformer module ability has been developed (as planned in Add Transformer ability to modules #38454)
   • In the mean time, the blocking synchronization can be worked around with an explicit "Transcriber" module that uses the stream::SynchronizingEDProducer (i.e. ExternalWork module ability) as in Integration of alpaka and macro-based SoA in CMSSW #38855.
2. The system is limited to one device per backend, because of the EventSetup model introduced here.
   • This shortcoming will be improved in the future by making the framework aware of different memory and execution spaces (to some degree).
   • For the mean time, I'll craft a temporary solution that would wrap the ESProducts for all devices into one ESProduct (similarly to the current model)
3. The operations launched in ESProducer are synchronized in a blocking way before the consuming modules are run
   • This will be improved in a future PR that will add a functionality similar to ExternalWork to ESProducers (Add ExternalWork-like mechanism for ESProducers #24185)
   • In the mean time I'd expect the impact of these synchronizations to be minimal in production
     • The current CUDA system also leads to device-host synchronizations around IOV boundaries because of the memory allocation patterns (as the caching allocator is not used).
     • As far as I've understood, at the HLT the Records that have device ESProducts have IOVs that much longer than a single lumi, which limits the main impact to the beginning of the jobs
     • The transfers are initiated earlier and concurrently

The code has many TODO: comments of which most are meant for further evolution after this PR. I'm planning to address the following in this PR

• Temporary mitigation ESProducts to be transferred for "every device" of a backend
• Look into replacing most/all #ifdef for backend-specific behavior with if constexpr (or equivalent)
• Few files that depend at most on Alpaka and FWCore/Utilities and are specific to CMS are currently placed in HeterogeneousCore/AlpakaCore and HeterogeneousCore/AlpakaInterface. They should be moved out from the former, but the latter seems non-ideal as well. New package?
• Finalize the naming convention
  • I'm specifically thinking the placement of Device in the names, currently they are along DeviceEvent, EDDeviceGetToken, global::EDProducer (i.e. no Device in name).
    • I wonder if there could be confusion between edm::global::EDProducer and ALPAKA_ACCELERATOR_NAMESPACE::global::EDProducer (where only the global::EDProducer part is visible in code)
    • Alternative could be along DeviceEvent, DeviceEDGetToken, global::DeviceEDProducer
    • Or device::Event, device::EDGetToken, device::global::EDProducer (all still enclosed in ALPAKA_ACCELERATOR_NAMESPACE)
• Add README.md documenting the interface (descoped from this PR)

PR validation:

Added unit test passes on both CPU and GPU.